Robust Scene Estimation for Goal-directed Robotic Manipulation in Unstructured Environments

To make autonomous robots "taskable" so that they function properly and interact fluently with human partners, they must be able to perceive and understand the semantic aspects of their environments. More specifically, they must know what objects exist and where they are in the unstructured human world. Progresses in robot perception, especially in deep learning, have greatly improved for detecting and localizing objects. However, it still remains a challenge for robots to perform a highly reliable scene estimation in unstructured environments that is determined by robustness, adaptability and scale. In this dissertation, we address the scene estimation problem under uncertainty, especially in unstructured environments. We enable robots to build a reliable object-oriented representation that describes objects present in the environment, as well as inter-object spatial relations. Specifically, we focus on addressing following challenges for reliable scene estimation: 1) robust perception under uncertainty results from noisy sensors, objects in clutter and perceptual aliasing, 2) adaptable perception in adverse conditions by combined deep learning and probabilistic generative methods, 3) scalable perception as the number of objects grows and the structure of objects becomes more complex (e.g. objects in dense clutter). Towards realizing robust perception, our objective is to ground raw sensor observations into scene states while dealing with uncertainty from sensor measurements and actuator control . Scene states are represented as scene graphs, where scene graphs denote parameterized axiomatic statements that assert relationships between objects and their poses. To deal with the uncertainty, we present a pure generative approach, Axiomatic Scene Estimation (AxScEs). AxScEs estimates a probabilistic distribution across plausible scene graph hypotheses describing the configuration of objects. By maintaining a diverse set of possible states, the proposed approach demonstrates the robustness to the local minimum in the scene graph state space and effectiveness for manipulation-quality perception based on edit distance on scene graphs. To scale up to more unstructured scenarios and be adaptable to adversarial scenarios, we present Sequential Scene Understanding and Manipulation (SUM), which estimates the scene as a collection of objects in cluttered environments. SUM is a two-stage method that leverages the accuracy and efficiency from convolutional neural networks (CNNs) with probabilistic inference methods. Despite the strength from CNNs, they are opaque in understanding how the decisions are made and fragile for generalizing beyond overfit training samples in adverse conditions (e.g., changes in illumination). The probabilistic generative method complements these weaknesses and provides an avenue for adaptable perception. To scale up to densely cluttered environments where objects are physically touching with severe occlusions, we present GeoFusion, which fuses noisy observations from multiple frames by exploring geometric consistency at object level. Geometric consistency characterizes geometric compatibility between objects and geometric similarity between observations and objects. It reasons about geometry at the object-level, offering a fast and reliable way to be robust to semantic perceptual aliasing. The proposed approach demonstrates greater robustness and accuracy than the state-of-the-art pose estimation approach.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/163060/1/zsui_1.pd

Semantic Robot Programming for Goal-Directed Manipulation in Cluttered Scenes

We present the Semantic Robot Programming (SRP) paradigm as a convergence of robot programming by demonstration and semantic mapping. In SRP, a user can directly program a robot manipulator by demonstrating a snapshot of their intended goal scene in workspace. The robot then parses this goal as a scene graph comprised of object poses and inter-object relations, assuming known object geometries. Task and motion planning is then used to realize the user's goal from an arbitrary initial scene configuration. Even when faced with different initial scene configurations, SRP enables the robot to seamlessly adapt to reach the user's demonstrated goal. For scene perception, we propose the Discriminatively-Informed Generative Estimation of Scenes and Transforms (DIGEST) method to infer the initial and goal states of the world from RGBD images. The efficacy of SRP with DIGEST perception is demonstrated for the task of tray-setting with a Michigan Progress Fetch robot. Scene perception and task execution are evaluated with a public household occlusion dataset and our cluttered scene dataset.Comment: published in ICRA 201

Influence of planting methods and organic amendments on rice yield and bacterial communities in the rhizosphere soil

A comprehensive understanding of rice cultivation techniques and organic amendments affecting soil quality, enzyme activities and bacterial community structure is crucial. We investigated two planting methods (direct seeding and transplanting) of paddy rice (Oryza sativa) and organic amendments with rice straw and biochar on crop yield and soil biological and physicochemical properties. Rhizosphere bacterial communities at the maturity stage of rice growth were characterized through high-throughput 16S rRNA sequencing. Soil biochemical properties and enzyme activity levels were analyzed. Grain yield of paddy rice with transplanting increased 10.6% more than that with direct seeding. The application of rice straw increased grain yield by 7.1 and 8.2%, more than with biochar and the control, respectively. Compared to biochar and the control, the application of rice straw significantly increased sucrase, cellulase, protease, organic carbon, available phosphorus, nitrate, and ammonium. The application of biochar increased microbial biomass nitrogen and carbon, urease, pH, available nitrogen, and available potassium compared to the application of rice straw and the control. Principal coordinate analysis and dissimilarity distances confirmed significant differences among the microbial communities associated with planting methods and organic amendments. Bacteroidetes, Nitrospirae, Firmicutes, and Gemmatimonadetes abundance increased with rice straw relative to biochar and the control. The biochar addition was associated with significant increases in Chloroflexi, Patescibacteria, Proteobacteria, and Actinobacteria abundance. Pearson’s correlation analyzes showed that Chloroflexi, Bacteroidetes and Nitrospirae abundance was positively correlated with grain yield. The relative abundance of these bacteria in soil may be beneficial for improving grain yield. These results suggest that planting methods and organic amendments impact soil biochemical characteristics, enzyme activity levels, and microbial community composition

Broad-Spectrum Antiviral Activity of RNA Interference against Four Genotypes of Japanese Encephalitis Virus Based on Single MicroRNA Polycistrons

Japanese encephalitis virus (JEV), a neurotropic mosquito-borne flavivirus, causes acute viral encephalitis and neurologic disease with a high fatality rate in humans and a range of animals. Small interfering RNA (siRNA) is a powerful antiviral agent able to inhibit JEV replication. However, the high rate of genetic variability between JEV strains (of four confirmed genotypes, genotypes I, II, III and IV) hampers the broad-spectrum application of siRNAs, and mutations within the targeted sequences could facilitate JEV escape from RNA interference (RNAi)-mediated antiviral therapy. To improve the broad-spectrum application of siRNAs and prevent the generation of escape mutants, multiple siRNAs targeting conserved viral sequences need to be combined. In this study, using a siRNA expression vector based on the miR-155 backbone and promoted by RNA polymerase II, we initially identified nine siRNAs targeting highly conserved regions of seven JEV genes among strains of the four genotypes of JEV to effectively block the replication of the JEV vaccine strain SA14-14-2. Then, we constructed single microRNA-like polycistrons to simultaneously express these effective siRNAs under a single RNA polymerase II promoter. Finally, these single siRNAs or multiple siRNAs from the microRNA-like polycistrons showed effective anti-virus activity in genotype I and genotype III JEV wild type strains, which are the predominant genotypes of JEV in mainland China. The anti-JEV effect of these microRNA-like polycistrons was also predicted in other genotypes of JEV (genotypes II and IV), The inhibitory efficacy indicated that siRNAs×9 could theoretically inhibit the replication of JEV genotypes II and IV

Physically Plausible Scene Estimation for Manipulation in Clutter

International audiencePerceiving object poses in a cluttered scene is a challenging problem because of the partial observations available to an embodied robot, where cluttered scenes are especially problematic. In addition to occlusions, cluttered scenes have various cases of uncertainty due to physical object interactions, such as touching, stacking and partial support. In this paper, we discuss these cases of physics-based uncertainty case by case and propose methods for physically-viable scene estimation. Specifically, we use Newtonian physical simulation to check the plausibility of hypotheses within generative probabilistic inference in relation to particle filtering, MCMC and an MCMC variant on particle filtering. Assuming that object geometries are known, we estimate the scene as a collection of object poses, and infer a distribution over the state space as well as the maximu likelihood estimate. We compare with ICP based approaches and present our results for scene estimation in isolated cases of physical object interaction as well as multiobject scenes such that manipulation of graspable objects can be performed with a PR2 robot